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METHOD OF PERFORMING VIDEO ENCODING RATE CONTROL USING BIT 

BUDGET 

RELATED APPLICATIONS 

This patent application is related to concurrently filed US Patent Application 

Serial No. , titled "Method of Performing Video Encoding Rate 

Control", by Kim et al. (attorney docket no. 042390.P10264), filed on 

, and to concurrently filed US Patent Application Serial No. 

, titled "Method of Performing Video Encoding Rate Control Using 

Motion Estimation", by Kim et al. (attorney docket no. 042390.p10265), filed on 

, both assigned to the assignee of the present invention and 

herein incorporated by reference. 

BACKGROUND 

The present disclosure is related to rate control of the encoding of video images. 

As is well-known, video encoding may be performed by any one of a number or 
variety of techniques. Common techniques that are frequently employed comply with 
certain established standards, such as the 'MPEG' (Moving Pictures Expert Group) and 
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'H.26x' standards. These include the following: ITU-T "Video coding for low bit-rate 
communications," ITU-T Recommendation H.263, version 1, Nov. 1995 and version 2, 
Jan. 1998; "Generic Coding of Moving Pictures and Associated Audio Information: 
Video," ISO/IEC 13818-2: International Standard 1995; and "Coding of audio-visual 
Objects-Part 2: Visual Amendment 1; Visual extensions," ISO/IEC 14496-2: Draft of 
January 6, 2000; respectively, referred to specifically as H.263, H.263+, MPEG-2, and 
MPEG-4 and generally as MPEG and H.26x, hereinafter. Such standards, however, 
define bit stream syntax so that any standard compliant decoder may be employed to 
decode the encoded video. This provides encoders with a relatively large amount of 
flexibility in terms of implementation. 

Rate control, such as bit rate control, is one of the issues not generally specified 
for the video encoder, therefore, making it possible to provide the capability to employ a 
variety of different techniques. Furthermore, applying rate control may impact the 
processing of video in several respects. In one respect, rate control may be employed 
to maintain buffer constraints and, thereby, prevent overflow and/or underflow during 
encoding and, in particular, in connection with real-time applications. Likewise, in 
another respect, rate control may also impact picture quality. 

In addition to rate control, there are other parameters that may be varied by an 
encoder during encoding that may result in various levels of image distortion, and, 
therefore, impact performance. Therefore, it may, at times, be difficult to improve the 
image quality to an acceptable level while also meeting various desired constraints, 
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such as, for example, a total bit budget, employing an appropriate amount of delay, etc. 



Techniques exist to balance these considerations in order to provide acceptable 
or desirable solutions. Examples include employing Lagrangian optimization or 
dynamic programming. Unfortunately, however, such techniques are frequently or 
typically computationally complex and, therefore, expensive in terms of the amount of 
processing resources consumed by such an approach. Furthermore, the dependency 
that typically exists between images or image frames in the processing of video may, at 
times, make addressing such issues even more complex. For example, the distortion of 
the current frame may depend at least in part on the selection of quantization 
parameters, for example, for the previous frame or frames. Therefore, a technique for 
performing rate control in video encoding that is less computationally complex than 
previous or state of the art approaches, but that also balances at least some of the 
foregoing complex considerations, is desirable. 



BRIEF DESCRIPTION OF THE DRAWINGS 



The subject matter regarded as the invention as particularly pointed out and 
distinctly claimed in the concluding portion of the specification. The invention, however, 
both as to organization and method of operation, together with objects, features, and 
advantages thereof, may best be understood by reference to the following detailed 
description when read with the accompanying drawings in which: 
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FIG. 1 is block diagrarri illustrating a video encoder that may employ an 
embodiment of a method of performing video encoding rate control using bit budget in 
accordance with the present invention; 

FIG. 2 is a series of plots illustrating the relationship between macroblock SAD 
and bit count for various picture types for a specific number of quantization bins; and 

FIG. 3 is a table illustrating the comparison of performance parameters between 
the Q2 control approach and an embodiment of a method of performing video encoding 
rate control using bit budget in accordance with the present invention. 

DETAILED DESCRIPTION 

In the following detailed description, numerous specific details are set forth in 
order to provide a thorough understanding of the invention. However, it will be 
understood by those skilled in the art that the present invention may be practiced 
without these specific details. In other instances, well-known methods, procedures, 
components and circuits have not been described in detail so as not to obscure the 
present invention. 

As previously described, video encoding rate control may be a feature of a video 
encoder. Although the invention is not limited in scope in this respect, in one 
embodiment of a method of performing video encoding rate control using bit budget in 
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accordance with the present invention, the video bit rate employed during video 
encoding is varied based at least in part on a measurement of the variation in pixel 
signal level values for a selected portion of a video image being encoded. One 
rationale for employing such an approach may be that the larger the range or variation 
in pixel signal level values, the greater the number of bits employed to represent or 
differentiate the different pixel signal value levels and vice versa. 

Therefore, for this particular embodiment, although, of course, the invention is 
not limited in scope in this respect, a relationship may be employed between the 
variation in pixel signal value levels for a macroblock and the associated video 
encoding rate control to be applied by the video encoder. Furthermore, although there 
are a variety of techniques that may be employed to measure variation and the 
invention is not limited in scope to any particular technique, in this particular 
embodiment, the variation may be measured using the sum of absolute differences 
(SAD), which is employed in motion estimation. 

SAD =min]TE \C[iJ]-R[x 0 +x + i 9 y 0 +y + j]\ [1] 
where 

upper left corner coordinates of the current macroblock 
current macroblock luminance samples 
reconstructed previous frame luminance samples 

search range: {(x,y):-\6 <x ? >><16 } 
As is well-known, the SAD values are computed in all or some selected search points in the 



(x 0 , Vo) 
C[x, yj 

R[x, y] 
S 
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search space (S). The motion vector {MVx, MVy) is selected based on the displacement of the 
search point which results in the minimum SAD among all the SAD values in the search space. 
It is noted, of course, that other potential measures of variation are essentially a 
substitute of the SAD. For example, the mean absolute difference (MAD) may be 
employed in place of the SAD and should provide nearly identical, if not identical, 
results. Therefore, such other substitute measures are clearly within the scope of the 
invention. 

In this context, the SAD provides several advantages. It is already computed as 
part of motion estimation, and, therefore, introduces little or no additional overhead in 
terms of the consumption of processing resources. Furthermore, motion estimation 
provides information that may be useful in terms of video encoding rate control. For 
example, motion estimation provides information about prediction mode decisions, 
motion vector choices, and displaced frame difference coding fidelity. 

In this context, it is noted that a modification in quantization step size specifically 
results in an adjustment of a video encoding rate, here the video encoding bit rate. 
Therefore, although, again, the invention is not limited in scope in this respect, for this 
embodiment, adjusting the quantization step size is a mechanism employed to modify 
or adjust the video encoding rate. This follows at least in part from the observation that 
a high quantization step size provides relatively coarse quantization. Thus, the amount 
of information to be sent to the decoder is reduced when employing a high quantization 
step size. 
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It may in this context be desirable to appropriately characterize the relationship 
between the bit count employed to encode a macroblock and the SAD of the 
macroblock, at least for this particular embodiment. Therefore, for different values of a 
quantization step size parameter, here from one to 31, these particular parameters are 
computed for a variety of images. Of course, this is just one potential methodology and 
any one of a number of methodologies may be employed. The invention is not limited 
in scope to employing any particular methodology. Therefore, furthermore, in this 
particular embodiment, as shall be described in more detail hereinafter, the 
macroblocks (MBs) are classified by type, such as inter, intra, B and 4 MV. In this 
context, Intra' refers to a MB coded without motion vectors, 'inter' refers to a MB that 
uses one forward motion vector, '4 MV refers to a MB that uses four forward motion 
vectors, and B refers to a MB that uses forward and backward motion vectors to reduce 
temporal redundancy, although, again, the invention is not limited in scope in this 
respect. It is noted that the modes also provide information based on motion estimation 
that may be useful in video encoding rate control. 

In this particular embodiment, although, again the invention is not limited in 
scope to employing this particular methodology, the SAD is obtained after motion 
estimation has been performed, such as at the point shown in the block diagram 
illustrated in FIG. 1, except for intra macroblocks, of course. This point in FIG. 1 is 
chosen so that the mode of each macroblock using the results of motion estimation 
may be obtained for this particular embodiment. 
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Using this methodology or approach, a relationship between macroblock SAD 
and the count may be generated for each quantization parameter or step-size. In this 
particular implementation, based upon the quantization parameter or step-size, 31 
figures may, therefore, be generated, although this is not intended to be limiting on 
alternate approaches within the scope of the present invention. Here, then, for each 
different quantization step-size, from a number of macroblocks having SADs, the total 
number of bits is determined. Likewise, as previously described, the different 
macroblock types may also be employed. The relationship between SAD and bit count 
may be shown to depend at least in part on type of macroblock, in addition to 
depending at least in part on the SAD of the macroblock, at least for this particular 
implementation; however, as previously indicated, the invention is not limited in scope 
to this particular implementation. 

In order to make the data generated suitable for use in video encoding rate 
control, it is desirable to quantize the macroblock SAD, although, again, of course, the 
invention is not limited in scope in this respect. For example, some other embodiments 
may employ the foregoing approach regarding SAD without applying quantization. 
Nonetheless, the following quantization technique is employed in this particular 
embodiment. Of course, any one of a number of other suitable techniques may 
alternatively be employed, and all such other quantization techniques are included 
within the scope of the present invention because the particular technique applied is not 
significant. However, in this particular embodiment or methodology, the following 
quantization technique is employed. 
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index = SAD/bin_size 

[2] 

where bin_size = range/no_bins 

In equations [2], 'SAD 1 is, of course, the macroblock SAD. Likewise, 'no_bins' is 8. For 
a given quantization step-size, in this embodiment, it is then desirable to average the bit 
count depending upon the particular index. This is illustrated by the plots in FIG. 2 
depending on the picture types employed, in this implementation, types I, P, or B. It is 
noted that such picture types are employed in connection with MPEG compliant video 
encoders, although, again, the invention is not limited in scope to MPEG or compliance 
with MPEG. 

The plots shown in FIG. 2 illustrate on one graph the relationship between SAD 
and bit count where quantization step-size is held constant for each separate curve, but 
varied across the family of curves shown on each respective plot. Using this data, 
therefore, for a video encoder, the video bit rate to be employed may be varied to take 
into account the SAD of a macroblock, and/or the macroblock type. More specifically, 
by quantizing the total number of bits and the macroblock SAD, the plots as shown in 
FIG. 2 may be converted into lookup tables (LUTs) that may be stored and employed by 
a video encoder during the process of encoding video to apply video encoding rate 
control. For these plots, 40 bins were employed for bit count for each picture type, 
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For such an embodiment, the following methodology may be employed, 
although, again, the invention is not limited in scope in this respect. For an image or 
frame, a maximum acceptable quantization step-size is employed. Then, using the 
index, here quantized SAD, and the quantization step-size, from the look- up tables the 
rate, or alternatively the bit count, is computed for the frame or image. Here, this is 
done on a macroblock basis and then summed over the image or frame, although, of 
course, the invention is not limited in scope in this respect. If the rate or bit count that is 
computed for the frame is less than the rate indicated by the bit budget, this indicates 
that the quantization step-size may be lowered to provide better performance without 
exceeding the budget. Therefore, the quantization step is lowered and the previously 
described process is repeated until the bit budget or rate is exceeded, indicating that 
the quantization step-size reduction has reached a limit. 

In this particular embodiment, the maximum acceptable step-size is 20 for I and 
P type pictures and 28 for B type pictures. Therefore, for this embodiment, the 
following pseudo-code may be employed to implement this embodiment. 



For (l=Qp_max; 1>= Qp_min; 1--) { 
sum_rate = 0; 
for (i=0; i<N; i++) { 

for (j=0; j<M; j { 

sum_rate += RATE [index] [1] ; 

} 

} 

if (sum_rate>budget) break; 

} 

Qp - 1; /* your desired Qp for current frame */ 
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N, M are image height and width divided by 16. NxM represents number of 
macroblocks per frame. Sum_rate represents the estimated bits spent using the 
index and Qp relationship. Budget is the allocated bits for the current frame. 

In comparison with state of the art approaches to adjusting the video bit rate, an 
embodiment in accordance with the present invention has several advantages. For 
example, an approach referred to as Q2 is used in connection with MPEG-4. The 
target bit rate is computed based on the bits available and the last encoded frame. If 
the last frame is complex and uses excessive bits, more bits might be assigned to this 
frame. However, if there are fewer bits left for encoding, fewer bits might be assigned 
due to the bit budget. Therefore, a weighted average provides a compromise between 
these two factors. 

Once the target bit rate for the frame is determined, the quantization step-size to 
meet it is selected. This is accomplished using a least squares statistical modeling 
technique. The encoder rate distortion function is modeled as: 

R = X r -^X 2 ~ [3] 
1 Q 2 Q 2 

The encoding bit count is denoted R. The encoding complexity, denoted S, is 
measured using the mean absolute difference (MAD). The quantization step-size is the 
parameter Q. The modeling parameters, Xi and X 2 , are estimated using least squares 
from previous data. Then the equation above is solved for Q. To solve the equation 
using this technique, typically up to 20 previous frames of data are employed, 
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suggesting computation complexity as well as employing significant memory. 
Furthermore, simulation results indicate that the Q2 technique does not meet the bit 
budget, that is, comply with the target rate, for all images. 

One advantage of this particular embodiment, therefore, is reduced 
computational complexity. For this particular embodiment, for example, one parameter, 
bit budget or rate is employed. Likewise, in this embodiment, although again the 
invention is not limited in scope in this respect, is macroblock SAD is employed. In 
terms of computational complexity, this does not produce a significant amount of 
additional overhead because the SAD is calculated to determine macroblock mode, as 
previously described, except for I frames. Furthermore, this computation for I frames, 
although providing some additional overhead, is not significant in terms of the 
processing resources that are consumed. Likewise, as the results below indicate, the 
bit budget is met for all images. 

FIG. 3 is a table providing a comparison between various performance 
parameters for an embodiment in accordance with the invention and Q2. This data was 
generated from six image sequences. 150 frames from each was employed with a 
frame rate of 15 frame per second. The number of B frames between P or I frames is 2 
and the intra period is 15 frames. The data in the table implies that the degradation in 
performance quality is slight, and in some cases is better. Furthermore, this 
embodiment stayed within budget with greater compression efficiency. 
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The previously described embodiments provide a number of desirable 
advantages and features. For example, as previously explained, the implementation of 
a rate control mechanism, such as those previously described, reduces computation 
complexity. Therefore, although results may vary depending on a variety of factors, 
such embodiments may be suitable for low-power applications, as is often desirable. 
Likewise, the previously described embodiments may be implemented in hardware, 
software, firmware, or any combination thereof. Furthermore, embodiments in 
accordance with the invention provide compatibility with known video standards, such 
as MPEG and H.26x. 

It will, of course, be understood that, although particular embodiments have just 
been described, the invention is not limited in scope to a particular embodiment or 
implementation. For example, one embodiment may be in hardware, whereas another 
embodiment may be in software. Likewise, an embodiment may be in firmware, or any 
combination of hardware, software, or firmware, for example. Likewise, although the 
invention is not limited in scope in this respect, one embodiment may comprise an 
article, such as a storage medium. Such a storage medium, such as, for example, a 
CD-ROM, or a disk, may have stored thereon a look up table, such as previously 
described. Likewise, a storage medium may have stored instructions, which when 
executed by a system, such as a computer system or platform, or an imaging system, 
for example, may result in an embodiment of a method in accordance with the present 
invention being executed, such as an embodiment of a method of performing video 
encoding rate control using bit budget, for example, as previously described. For 
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example, a video processing platform or an imaging system may include a video 
encoder, a video input device and memory. The video encoder may include a 
mechanism to adjust the video encoding rate employed during video encoding, such as 
by employing one of the embodiments previously described, for example. Furthermore, 
embodiments of the invention are also not limited to video encoders or video encoding. 
For example, video may be decoded where the video had been encoded using an 
embodiment in accordance with the invention, again, such as previously described. 

While certain features of the invention have been illustrated and described 
herein, many modifications, substitutions, changes and equivalents will now occur to 
those skilled in the art. It is, therefore, to be understood that the appended claims are 
intended to cover all such modifications and changes as fall within the true spirit of the 
invention. 
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